Automated indexing and extraction of multiple information fields in digital records

ABSTRACT

Systems and Methods are disclosed herein for automatically indexing multiple informational fields in digital data records, the method comprising: identifying, based on rules defining target information fields, for each target field of the target information fields, at least one page in a digital data record comprising content related to the target field; extracting, for each target field, from the identified at least one page, at least one portion of text comprising the content; feeding, for each target field, a pre-processed version of the at least one portion of text into a machine learning (ML) model, wherein the ML model is trained on the target field; determining, for each target field, via the ML model trained on the target field, at least one candidate text comprising the content; and extracting, for each target field, the at least one candidate text.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority benefit from, and is a Continuation-in-part of U.S. application Ser. No. 18/098,055, Titled “AUTOMATED INDEXING AND EXTRACTION OF INFORMATION IN DIGITAL DOCUMENTS”, and filed on Jan. 17, 2023, which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

Disclosed are automated systems and methods to index information in digital documents, and data records, which in various instances rely on trained machine learning networks individually or in combinations with other configured modules, devices, or processes. In particular, this application is directed to automated standard indexing of multiple information fields in digital data records.

SUMMARY

In numerous aspects, disclosed herein is a method to automatically index multiple informational fields in a digital data records, the method comprising: identifying, based on rules defining target information fields, for each target field of the target information fields, at least one page in a digital data record comprising content related to the target field; extracting, for each target field, from the identified at least one page, at least one portion of text comprising the content; feeding, for each target field, a pre-processed version of the at least one portion of text into a machine learning (ML) model, wherein the ML model is trained on the target field; determining, for each target field, via the ML model trained on the target field, at least one candidate text comprising the content; and extracting, for each target field, the at least one candidate text.

In numerous aspects, disclosed herein is a system to automatically index multiple informational fields in digital data records, the system comprising: a plurality of trained NLP ML models, wherein each ML model of the plurality of trained NLP models is trained on a specific target information field of a plurality of target information fields; a database to store indexed information; a memory comprising instructions, coupled to a processor, wherein the instructions are executable by a processor to: identify, based on rules defining the target information fields, for each target field of the target information fields, at least one page in the digital data record comprising content related to the target field; extract, for each target field, from the identified at least one page, at least one portion of text comprising the content; feed, for each target field, a pre-processed version of the at least one portion of text into an ML model of the plurality of trained NLP ML models, wherein the ML model is trained on the target field; determine, for each target field, via the ML model trained on the target field, at least one candidate text comprising the content; extract, for each target field, via the ML model trained on the target field, the at least one candidate text; and generate for each target field, via the ML model trained on the target field, an index of the at least one candidate text.

In numerous aspects, disclosed herein is a system to automatically index multiple informational fields in digital data records, the system comprising: a plurality of trained NLP ML models, wherein each ML model of the plurality of trained NLP models is trained on a specific target information field of a plurality of target information fields; a database to store indexed information; a memory comprising instructions, coupled to a processor, wherein the instructions are executable by a processor to identify, based on rules defining the target information fields, for each target field of the target information fields, at least one page in the digital data record comprising content related to the target field; extract, for each target field, from the identified at least one page, at least one portion of text comprising the content; feed, for each target field, a pre-processed version of the at least one portion of text into an ML model of the plurality of trained NLP ML models, wherein the ML model is trained on the target field; determine, for each target field, via the ML model trained on the target field, at least one candidate text comprising the content; extract, for each target field, via the ML model trained on the target field, the at least one candidate text; and generate for each target field, via the ML model trained on the target field, an index of the at least one candidate text.

In numerous aspects, disclosed herein is a non-transitory computer-readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for providing a sentiment for digital information comprising receiving a digital data record; identifying, based on rules defining target information fields, for each target field of the target information fields, at least one page in the digital data record comprising content related to the target field; extracting, for each target field, from the identified at least one page, at least one portion of text comprising the content; feeding, for each target field, a pre-processed version of the at least one portion of text into a machine learning (ML) model, wherein the ML model is trained on the target field; determining, for each target field, via the ML model trained on the target field, at least one candidate text comprising the content; and extracting, for each target field, the at least one candidate text.

BRIEF DESCRIPTION OF THE DRAWINGS

In the description, for purposes of explanation and not limitation, specific details are set forth, such as particular aspects, procedures, techniques, etc. to provide a thorough understanding of the present technology. However, it will be apparent to one skilled in the art that the present technology may be practiced in other aspects that depart from these specific details.

The accompanying drawings, where like reference numerals refer to identical or functionally similar elements throughout the separate views, together with the detailed description below, are incorporated in and form part of the specification, and serve to further illustrate aspects of concepts that include the claimed disclosure and explain various principles and advantages of those aspects.

The systems, and methods disclosed herein have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the various aspects of the present disclosure so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.

FIG. 1 illustrates a flow chart of one aspect of a method to automatically index and extract specialized tabulated information in a document, according to at least one aspect of the present disclosure.

FIG. 2 illustrates a flow diagram of one aspect of a method to automatically index specialized targeted information in a digital data record, according to at least one aspect of the present disclosure.

FIG. 3 illustrates a flow chart of one aspect of a method to train multiple types of machine learning networks to autonomously identify targeted information, according to at least one aspect of the present disclosure.

FIG. 4 presents a block diagram of a computer apparatus, according to at least aspect of the present disclosure.

FIG. 5 is a diagrammatic representation of an example system that includes a host machine within which a set of instructions to perform any one or more of the methodologies discussed herein may be executed, according to at least one aspect of the present disclosure.

FIG. 6 illustrates one example of the inputs and outputs of automated page selection from a digital data record, according to at least one aspect of the present disclosure.

FIG. 7A-7B illustrate one example of the inputs and outputs of image generation from a digital data record, according to at least one aspect of the present disclosure.

FIG. 8 illustrates one example of the inputs and outputs of extraction of sections of an image, according to at least one aspect of the present disclosure.

FIG. 9 illustrates one example of the inputs and outputs of extraction of data, information, or text from extracted portions of an image, according to at least one aspect of the present disclosure.

FIG. 10 illustrates one example of the inputs and outputs of extraction of data, information, or text from extracted portions of an image, according to at least one aspect of the present disclosure.

FIG. 11 illustrates a flow chart of one aspect of a method to automatically extract indexing fields from a digital data record, according to at least one aspect of the present disclosure.

FIG. 12 illustrates a flow diagram of one aspect of a method to automatically extract indexing fields from a digital data record as well as extracting specialized information via a separate pipeline, according to at least one aspect of the present disclosure.

FIG. 13 illustrates a method of extracting and indexing content from a digital data record, according to at least one aspect of the present disclosure.

FIG. 14 illustrates one example of inputs and outputs of page selection of a digital data record, according to at least one aspect of the present disclosure.

FIG. 15 illustrates one example of inputs and outputs of text extraction of a digital data record, according to at least one aspect of the present disclosure.

FIG. 16 illustrates one example of inputs and outputs of preprocessing of text in a digital data record, according to at least one aspect of the present disclosure.

FIG. 17 illustrates one example of text inference output from preprocessed text in a digital data record, according to at least one aspect of the present disclosure.

FIG. 18 illustrates another example of text inference output from preprocessed text in a digital data record, according to at least one aspect of the present disclosure.

FIG. 19 illustrates one example of the inferences output by the methods disclosed herein to be combined and displayed, according to at least one aspect of the present disclosure.

DESCRIPTION

Before discussing specific embodiments, aspects, or examples, some descriptions of terms used herein are provided below.

As used herein, the term “computing device” or “computer device” may refer to one or more electronic devices that are configured to directly or indirectly communicate with or over one or more networks. A computing device may be a mobile device, a desktop computer, and/or the like. As an example, a mobile device may include a cellular phone (e.g., a smartphone or standard cellular phone), a portable computer, a wearable device (e.g., watches, glasses, lenses, clothing, and/or the like), a personal digital assistant (PDA), and/or other like devices. The computing device may not be a mobile device, such as a desktop computer. Furthermore, the term “computer” may refer to any computing device that includes the necessary components to send, receive, process, and/or output data, and normally includes a display device, a processor, a memory, an input device, a network interface, and/or the like.

As used herein, the term “server” may include one or more computing devices which can be individual, stand-alone machines located at the same or different locations, may be owned or operated by the same or different entities, and may further be one or more clusters of distributed computers or “virtual” machines housed within a datacenter. It should be understood and appreciated by a person of skill in the art that functions performed by one “server” can be spread across multiple disparate computing devices for various reasons. As used herein, a “server” is intended to refer to all such scenarios and should not be construed or limited to one specific configuration. The term “server” may also refer to or include one or more processors or computers, storage devices, or similar computer arrangements that are operated by or facilitate communication and processing for multiple parties in a network environment, such as the Internet, although it will be appreciated that communication may be facilitated over one or more public or private network environments and that various other arrangements are possible.

Reference to “a device,” “a server,” “a processor,” and/or the like, as used herein, may refer to a previously recited device, server, or processor that is recited as performing a previous step or function, a different server or processor, and/or a combination of servers and/or processors. For example, as used in the specification and the claims, a first server or a first processor that is recited as performing a first step or a first function may refer to the same or different server or the same or different processor recited as performing a second step or a second function.

As used herein, the term “system” may refer to one or more computing devices or combinations of computing devices (e.g., processors, servers, client devices, software applications, components of such, and/or the like).

The rise of the digitization of documents across all platforms and industries, and the movement away from manual indexing, identification, and sorting of paper-based documents has led to various technological solutions to parse, analyze, index, or extract information from digital data records. However, none of currently available technological solutions are able to identify, index, and extract targeted and complex information from a digitized document or data record based on what the user desires, especially if the text is highly variable and contains unpredictable features. While searching or matching key words or phrases in digital data records is common, indexing targeted complex information is difficult when using different types of documents or data records, for example text and image-based documents, and where the information is within various different structures, for examples within tables, under sections, in cells, free text and the like. The combination of all these factors makes it difficult for autonomous technical solutions to be able to index sought or targeted information by a user.

Specifically sought or targeted complex information may be related to a specific industry. For example, product manufacturing compliance regulatory landscapes have become increasingly stringent, driving chemical producers, distributors, and users to align their practices more with the principles of green chemistry, with an aim to reduce or eliminate the use or generation of hazardous substances across the life cycle of a chemical product. A system to manage and access such information relies on information available in safety data sheets (SDS) that may be in a digitized format. However the complex nature of chemical ingredient information, and the fact that chemical ingredients and compositions are made up of various structures, alphanumeric characters, symbols, and associations with various variables and factors, as well as the fact that such information in safety data sheets is structured in various different ways, and is highly variable, for example, target information may fall under specific sections, or within tables that could exist anywhere on these SDSs, makes it technically very difficult to automate the indexing of these documents, identify relevant information, extract or analyze target information.

Furthermore, a rule-based or solely rule-based information extraction and identification system is not sufficient in complex industries and for complex documents, such as SDS documents produced by stakeholders in the chemical industry. This could, for example, be because each product manufacturer follows its own template and design for its SDS or digital data records. For certain manufacturers, this information may be presented in a non-complex or in a straightforward manner, while for many others it may be in a complex and diverse manner. The inconsistencies across manufacturers and the need to extract information from an SDS that may be a combined document produced as a result of collaboration by multiple actors or manufacturers dictates the need to have an intelligent machine learning system capable of extracting and identification system.

Disclosed herein are systems and methods that provide a technical solution to automate indexing, extracting, and identifying targeted information in complex data structures in documents, and scaling this automation to be usable for large volumes of complex data structures, including digital data records such as SDSs. An SDS document usually covers a chemical product, its name, and various details. The SDS is generally divided into multiple sections, usually 16, where a chemical ingredient or composition section is present in one of the sections. A chemical product covered by the SDS may be composed of a single ingredient that may be arranged in a tabular format.

In one aspect an automated system to index the composition information of products from Safety Data Sheets is presented. The system specifically indexes the ingredient names and their corresponding Chemical Abstracts Service (CAS) numbers and weight percentages. The number of ingredients in a chemical product and the presence or absence of the corresponding CAS numbers and weight percentages are unknown variables of the composition, and the system is designed to handle that variability. The system takes the SDS document (e.g., in PDF format) as the input and gives the list of ingredient names along with their corresponding CAS numbers and weight percentages in a tabular format as the output, which can be stored in a database or a file. In several aspects, the ingredient names and other details are structured in tabular format in the SDS documents. The system uses a combination of Machine Learning techniques (Computer Vision and Natural Language Processing) and rule-based systems that may be undertaken serially or in parallel.

In another aspect, while CAS numbers and weight percentages are being processed by one or more modules, other modules may be used to process other types of information in a SDS or digital data record, and these multiple processes are then combined to provide indexing of various outputs in digital data records that may then be stored and/or displayed to a user.

FIG. 1 illustrates a flow chart of one aspect of a system to automatically index and extract tabulated information in a document, according to at least one aspect of the present disclosure. System 100 may facilitate information to be requested or sought from a document 101, which may be a digital data record, of a text or image format, such as a PDF. Document 101 may also be an SDS. Document 101 may be text or image based. A user seeking specific information in document 101 (this sought specific information is also referred to herein as “targeted information” or “target information”) may input document 101 into a page selection module 103 that may execute one or more processes or threads, or be comprised of multiple other components or modules to select or identify one or more pages in document 101 where the targeted information resides. In various embodiments, page selection module 103 may be comprised of various disparate processes, which may be undertaken on an individual device, or on multiple devices, that select at least one page containing target information.

Page selection module 103 may in various aspects, be configured to identify page(s) within a document 101 that contains target information, for example chemical composition information in an SDS, and select these page(s), to be included in a list for example of all identified page numbers. In several aspects, to identify or select a page document 101 is parsed to extract the text from the document using extraction software such as PDF extraction software. In some aspects, this extracted text is cleaned and passed as a string of characters, alphanumerical text, and/or numbers to another stage in a pipeline or process executed by page selection module 103.

To identify a page in document 101 with target information, a combination of rules designed to determine the beginning and end sections in the text/string of characters, along with the presence of targeted information is used. This may include a combination of rules to identify composition of chemical ingredient information in document 101, when it is an SDS, along with the presence of the relevant ingredient information within an identified section. In particular, regular expressions, or expression matching may be used to identify section headers, chapters, and other identifiers of different sections in the text or strings of document 101. Regular expressions or expression matching may also be used to detect particular numbers or numerical patterns, or patterns of numbers, for example CAS numbers. Expression matching can also be used to detect known words, expressions, or phrases such as ingredient labels. All these various forms of expression matching may be used to identify various components of document 101.

Page selection module 103 may also contain a machine learning component. The machine learning (“ML”) network may be a natural language processing (“NLP”) machine learning model trained to detect specific names, words, symbols, phrases, alphanumerical combinations, or expressions. In one example the NLP model may be trained to detect the presence of chemical ingredient names. In one aspect, the NLP model used is a pre-trained Bidirectional Encoder Representations from Transformers (BERT) Named Entity Recognition (NER) model, fine-tuned on a custom dataset. In several examples the custom dataset may contain texts of the composition section of SDS documents. The model takes the text split into tokens as the input and is designed to identify three categories of tokens: a first word in a name, phrase, or expression, for example, the first word of the chemical ingredient name, the subsequent words of the name, phrase, or expression, such as a chemical ingredient name, and the words not belonging to the name, phrase or expression, such as the chemical ingredient name. In this step, the system uses the model to look for the presence of any tokens belonging to name, phrase, or expression, such as the example chemical ingredient name.

If the expression matching and/or NLP model fail to identify target information, for example chemical ingredient information, names, or compositions of chemicals in SDS documents or document 101, then pre-determined or pre-set rules may be used or autonomously implemented by the module 103 to find target information. Rules may be designed and configured for various configurations or documents. Depending on the target information sought, or the structure of the information, for example being in a table or unstructured text, or a list or otherwise, the rules that are implemented by page selection module may be altered accordingly. An example of rules that could be applied when target information includes chemical ingredient information may be rules configured to determine or check if the beginning and end of a chemical composition section is on the same page. If the text between the sections contains the relevant ingredient information, the page is added to the list for pages to be selected by module 103.

A pre-configured rule may also include to check or determine if only the beginning of a section is found on any page. If the text after this point contains the target information, such as relevant ingredient information, the page is added to the list of correct pages. A rule may also determine or check if only the end of a section is found on any page. If the text before this point contains the relevant or target information, for example ingredient or chemical information, the page is added to the list of correct pages. Finally, if all these fail, then a fall back search or one-off very specific identification matching query may be run, for example if no correct pages are identified, the module 103 determines or checks if a specific number or numerical pattern is present in a specific section, such a fallback rule may include determining if a CAS number is present in section 1 of a document 101. If yes, then page 1 is the correct page and is selected. Any combination of the methods and processes described above may be used by page selecting module 103 or by individual processes or devices to select a page with target information in document 101.

Once a page(s) is selected, in various aspects, the selected pages by module 103 are input into an image generation module 106 where image 107 is generated of each selected page. System 100 may continue by identifying location of sections, or depending on the type of document 101, a table, chart, list or other form of structured information or data, sought or other configuration specifying the type of data or data structure sought. In one aspect, a data structure or section identification module 108 may be or include a machine learning model component, for example a Computer Vision (“CV”) machine learning network trained for the purpose. The CV model may in several instances be a pre-trained Cascade Mask R-CNN object detection model, fine-tuned on a custom dataset containing images of SDS documents.

In several aspects the output of section identification module 108, or a component of it, for example CV network or model, receives an input of the image(s) 107 that was generated, for example by the image generation module 106, detects the structure with the target information, or the relevant section of the target information and outputs coordinates 109 of the sections in the image containing the target information. If the CV machine learning network or model is unable to identify the location or coordinates of sections containing target information, either because it fails to do so or because the data is not structured in the way the model was trained to detect, this could occur for example if a CV model is trained to detect tables or data in tables, but the information or data was not in a table but in another format such as sections, or free-flowing text. In these cases, coordinates 109 of sections containing the target information may be obtained by running optical character recognition (OCR) in addition to a combination of pre-set rules, which may be identical or at least similar to those applied by module 103 to identify target information in order to select the page. The coordinates 109 are in either case used as inputs in other parts of system 100.

System 100 may also comprise a table data or section extraction module 110 which relies on the input coordinates 109 to extract the tables from the provided image. In several aspects, document 101, identified or selected page numbers 104 in addition to coordinates 109 are input into a process pipeline or a table data or section extraction module 110 which by using the input coordinates 109 along with selected page numbers 104 may extract target information from the correction sections of the pages corresponding to the selected page numbers 104 of document 101. This extracted information may then be the output of the extraction module 110. In some aspects, the table data or section extraction module 110 extracts the target information in its original structure, as extracted data 111. In various aspects for example, a tabular structure is extracted and then converted into a 2D data structure such as a data frame. In other embodiments the whole section or tabular structure is extracted as extracted data 111. In several embodiments the target information is directly extracted as extracted data 111 from the ascertained coordinates 109 and pages 104 from document 101.

System 100 may then retain the structure of the target information in extracted data 111 as a 2D data frame or in another data structure format. The extracted data 111 is then input into a target information extraction module 112 that in various aspects maybe configured to be a chemical ingredient data extraction module. In several aspects the target information extraction module 112 outputs target information as data 113, which could be in any type of file, including a .csv file format. In numerous aspects, the extracted data 111 retains the tabular structure, which is used in the subsequent step to identify weight percentages of chemicals and to use these known associations between different parts of extracted data 111, for example the chemical names, CAS numbers, and weight percentages. This could for example be done by associating the weight percentages of chemicals to associate with previously identified or extracted weigh percentages.

System 100 may remove noise from extracted data 111, which may include steps to clean textual data. Associations in the cleaned data may then be used to identify specific target information by the target information extraction module 112. For example, if the system 100 is directed towards extracting and obtaining chemical ingredient information, it may use known associations between the data to determine ingredient names along with the corresponding CAS numbers and weight percentages. Alternatively, there may be no known associations, but associations are identified at the tabular structure, or other data structure, generally within the area of the identified page(s) that the CV model has selected. The extraction module may be comprised of an NLP Machine learning model, which may be the same or a different NLP model to the one used to select pages on in page selection module 103. In several aspects the NLP model may be a BERT NER model that identifies specific text, expressions, names, or phrases, for example chemical ingredient names in extracted data 111.

Tokens may be generated or the tokens generated at the page selecting process may be reused, for example, the text is split into three categories of tokens: the first word of a name, or phrase, such as a chemical ingredient name, the subsequent words of the name or phrase, such as the chemical ingredient name, and the words not belonging to the name or phrase, such as a chemical ingredient name. System 100 uses the model to look for the presence of any tokens belonging to a name or phrase such as a chemical ingredient name. The NLP model may make predictions for each token of the text individually, and additional post-processing rules are used to get the full names, such as a full name of a chemical ingredient instead of just abbreviations or symbols.

The table or data structure that was extracted may be scanned by system 100 or the NLP model to look for target name and identify the rows and columns containing them. Some columns may contain other information that are associated with the target names or phrases being sought, in the example of SDS documents, the columns (or rows in some aspects) containing CAS numbers and weight percentages are identified using regular expressions, or expression matching and the tabular structure of the data is used to determine an association between the chemical names, CAS numbers, and weight percentages. Multiple other associations or associated information may be inferred, determined or extracted from the information in the columns or rows. In the SDS document example, the regular expressions for CAS numbers are also designed to identify non-numerical values like trade secrets, mixture information and the like. The final output 113 is a 2D structure where each row corresponds to an ingredient, and the columns correspond to the different pieces of information belonging to an ingredient. This can be stored in a structured database or a file 113.

FIG. 2 illustrates a flow diagram of one aspect of a method 200 to automatically index targeted information in a digital data record, according to at least one aspect of the present disclosure. With reference now primarily to FIG. 2 together with FIG. 1 , in one aspect, method 200 may commence by selecting 205 a page number of a digital data record, for example document 101, FIG. 1 to identify a page containing targeted information. This in many aspects could occur for example via a page selecting module 103, FIG. 1 . The page corresponding to the selected page number may then be input 210 as an image into a visual machine learning network (visual ML), and/or into a visual detection module, for example section identification module 108, FIG. 1 . The visual ML may identify 215 a section of the image that contains the targeted information, the section may be identified by coordinates. Method 200 may then continue to input 220 the page number, the digital data record, and the coordinates of the identified section in an extraction module, for example table data or section extraction module 110, FIG. 1 . The extraction module may then extract 225 the targeted information from the section that was identified 215. In various aspects any of the processes, systems, or methods in system 100, FIG. 1 may be combined with method 200, and in any order or combination.

In several aspects, method 200 may also comprise inputting the extracted targeted information into a natural language processing ML network (NLP), that may for example be part of module 112, FIG. 1 , and identifying at least one data item, by the NLP network, based on a structure of the extracted targeted information. The data item may be of any type, and in the context of SDS documents may include for example chemical names, CAS numbers, and chemical structure weights. Similar to the system 100, FIG. 1 , the selecting 205 of a page number may be comprised of various processes and depending on the aspect may comprise parsing the digital data record to produce a string of characters and then identifying relevant portions in the string of characters containing the targeted information. Furthermore, the identification of these relevant portions may itself comprise splitting the produced string of characters into tokens which are input into a natural language processing ML network (NLP), and then identifying by the NLP a first word of a chemical ingredient name, a subsequent word of a chemical ingredient name, or a word not belonging to any chemical ingredient name.

In instances where the NLP fails to identify these names, then specific predetermined rules may be implemented to select page numbers containing targeted information. For example, these rules could include adding a page number to a list, once the system determines that both a beginning part and an end part of a relevant portion are on the same page. In other instances, a page number may be added to a list, if the system determines that a beginning part of a relevant portion is on a page, but not the end part of the relevant portion, and that the targeted information is on a subsequent page to the beginning part or section. A page number may also be added to a list if the end part of a relevant portion but not the beginning part is on a page, and the targeted information is on a previous page to the end part. Alternatively, if a specific number, for example a CAS number or other symbol or alphanumerical combination is on a page, the system may add that page to a list of relevant pages.

In several aspects, the relevant pages may include sections that have section headers, specific numbers, alphanumeric combinations, or keywords, wherein the identification is undertaken via expression matching in the string of characters. In several embodiments when a page number is identified, then an image is generated of the page corresponding to the page number in the document. This image may then be used as an input for example in a table or section detection module 108, FIG. 1 .

FIG. 3 illustrates a flow chart of one aspect of a method to train multiple types of machine learning networks to autonomously identify targeted information, according to at least one aspect of the present disclosure. In one aspect, method 300 commences with parsing 305 a custom dataset containing texts of digital data record to produce a string of characters. The custom dataset may be one curated specifically to train a machine learning network to identify specific information. For example, when training a machine learning network such as an NLP model to determine chemical names, CAS numbers, weightings and other information related to chemical ingredients, the custom dataset may be comprised of numerous SDS documents. Method 300 may then continue by identifying 310 portions in the string of characters containing information, and then split 315 the string into tokens that are then fed into a machine learning network for training. The processes 305-315 may be considered as preprocessing data in method 300 to prepare the training dataset. The machine learning network, which is an NLP model is then trained 320 by the tokens input into it. The training may comprise inputting the tokens into the NLP model; and outputting by the NLP model, a label where each of the tokens are identified or classified into a category. In one example embodiment, there may be three categories, and the token is categorized or classified into one of them. Example classifications or categories of tokens may be a first word of an ingredient name, subsequent word of an ingredient name, or not belonging to an ingredient name.

In several aspects, method 300 may continue with training a visual machine learning network (visual ML) such as a CV model on an image-based dataset, to recognize text or portions/sections of pages or images associated with pertinent information in an image. The pertinent information may be target information, such as chemical ingredient information. The training may comprise inputting image data from an image-based dataset into the visual ML; and outputting coordinates of identified relevant portions containing the pertinent information. The coordinates may include or border sections, tables, or other formatted information that is considered pertinent information. Once both the NLP model and the visual ML model are trained on provided datasets, then these models or networks may be utilized in any of the processes described above in relation to FIGS. 1-2 , and in any order or combination.

FIG. 4 is a block diagram of a computer apparatus 3000 with data processing subsystems or components, which a set of instructions to perform any one or more of the methodologies discussed herein may be executed, according to at least one aspect of the present disclosure. The subsystems shown in FIG. 4 are interconnected via a system bus 3010. Additional subsystems such as a printer 3018, keyboard 3026, fixed disk 3028 (or other memory comprising computer readable media), monitor 3022, which is coupled to a display adapter 3020, and others are shown. Peripherals and input/output (I/O) devices, which couple to an I/O controller 3012 (which can be a processor or other suitable controller), can be connected to the computer system by any number of means known in the art, such as a serial port 3024. For example, the serial port 3024 or external interface 3030 can be used to connect the computer apparatus to a wide area network such as the Internet, a mouse input device, or a scanner. The interconnection via system bus allows the central processor 3016 to communicate with each subsystem and to control the execution of instructions from system memory 3014 or the fixed disk 3028, as well as the exchange of information between subsystems. The system memory 3014 and/or the fixed disk 3028 may embody a computer readable medium.

FIG. 5 is a diagrammatic representation of an example system 4000 that includes a host machine 4002 within which a set of instructions to perform any one or more of the methodologies discussed herein may be executed, according to at least one aspect of the present disclosure. In various aspects, the host machine 4002 operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the host machine 4002 may operate in the capacity of a server or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The host machine 3002 may be a computer or computing device, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a portable music player (e.g., a portable hard drive audio device such as an Moving Picture Experts Group Audio Layer 3 (MP3) player), a web appliance, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

The example system 4000 includes the host machine 4002, running a host operating system (OS) 4004 on a processor or multiple processor(s)/processor core(s) 4006 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both), and various memory nodes 4008. The host OS 4004 may include a hypervisor 4010 which is able to control the functions and/or communicate with a virtual machine (“VM”) 4012 running on machine readable media. The VM 4012 also may include a virtual CPU or vCPU 4014. The memory nodes 4008 may be linked or pinned to virtual memory nodes or vNodes 4016. When the memory node 4008 is linked or pinned to a corresponding vNode 4016, then data may be mapped directly from the memory nodes 4008 to their corresponding vNodes 4016.

All the various components shown in host machine 4002 may be connected with and to each other or communicate to each other via a bus (not shown) or via other coupling or communication channels or mechanisms. The host machine 4002 may further include a video display, audio device or other peripherals 4018 (e.g., a liquid crystal display (LCD), alpha-numeric input device(s) including, e.g., a keyboard, a cursor control device, e.g., a mouse, a voice recognition or biometric verification unit, an external drive, a signal generation device, e.g., a speaker,) a persistent storage device 4020 (also referred to as disk drive unit), and a network interface device 4022. The host machine 4002 may further include a data encryption module (not shown) to encrypt data. The components provided in the host machine 4002 are those typically found in computer systems that may be suitable for use with aspects of the present disclosure and are intended to represent a broad category of such computer components that are known in the art. Thus, the system 4000 can be a server, minicomputer, mainframe computer, or any other computer system. The computer may also include different bus configurations, networked platforms, multi-processor platforms, and the like. Various operating systems may be used including UNIX, LINUX, WINDOWS, QNX ANDROID, IOS, CHROME, TIZEN, and other suitable operating systems.

The disk drive unit 4024 also may be a Solid-state Drive (SSD), a hard disk drive (HDD) or other includes a computer or machine-readable medium on which is stored one or more sets of instructions and data structures (e.g., data/instructions 4026) embodying or utilizing any one or more of the methodologies or functions described herein. The data/instructions 4026 also may reside, completely or at least partially, within the main memory node 4008 and/or within the processor(s) 4006 during execution thereof by the host machine 4002. The data/instructions 4026 may further be transmitted or received over a network 4028 via the network interface device 4022 utilizing any one of several well-known transfer protocols (e.g., Hyper Text Transfer Protocol (HTTP)).

The processor(s) 4006 and memory nodes 4008 also may comprise machine-readable media. The term “computer-readable medium” or “machine-readable medium” should be taken to include a single medium or multiple medium (e.g., a centralized or distributed database and/or associated caches and servers) that store the one or more sets of instructions. The term “computer-readable medium” shall also be taken to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by the host machine 4002 and that causes the host machine 4002 to perform any one or more of the methodologies of the present application, or that is capable of storing, encoding, or carrying data structures utilized by or associated with such a set of instructions. The term “computer-readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic media, and carrier wave signals. Such media may also include, without limitation, hard disks, floppy disks, flash memory cards, digital video disks, random access memory (RAM), read only memory (ROM), and the like. The example aspects described herein may be implemented in an operating environment comprising software installed on a computer, in hardware, or in a combination of software and hardware.

One skilled in the art will recognize that Internet service may be configured to provide Internet access to one or more computing devices that are coupled to the Internet service, and that the computing devices may include one or more processors, buses, memory devices, display devices, input/output devices, and the like. Furthermore, those skilled in the art may appreciate that the Internet service may be coupled to one or more databases, repositories, servers, and the like, which may be utilized to implement any of the various aspects of the disclosure as described herein.

The computer program instructions also may be loaded onto a computer, a server, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

Suitable networks may include or interface with any one or more of, for instance, a local intranet, a PAN (Personal Area Network), a LAN (Local Area Network), a WAN (Wide Area Network), a MAN (Metropolitan Area Network), a virtual private network (VPN), a storage area network (SAN), a frame relay connection, an Advanced Intelligent Network (AIN) connection, a synchronous optical network (SONET) connection, a digital T1, T3, E1 or E3 line, Digital Data Service (DDS) connection, DSL (Digital Subscriber Line) connection, an Ethernet connection, an ISDN (Integrated Services Digital Network) line, a dial-up port such as a V.90, V.34 or V.34bis analog modem connection, a cable modem, an ATM (Asynchronous Transfer Mode) connection, or an FDDI (Fiber Distributed Data Interface) or CDDI (Copper Distributed Data Interface) connection. Furthermore, communications may also include links to any of a variety of wireless networks, including WAP (Wireless Application Protocol), GPRS (General Packet Radio Service), GSM (Global System for Mobile Communication), CDMA (Code Division Multiple Access) or TDMA (Time Division Multiple Access), cellular phone networks, GPS (Global Positioning System), CDPD (cellular digital packet data), RIM (Research in Motion, Limited) duplex paging network, Bluetooth radio, or an IEEE 802.11-based radio frequency network. The network 4030 can further include or interface with any one or more of an RS-232 serial connection, an IEEE-1394 (Firewire) connection, a Fiber Channel connection, an IrDA (infrared) port, a SCSI (Small Computer Systems Interface) connection, a USB (Universal Serial Bus) connection or other wired or wireless, digital or analog interface or connection, mesh or Digi® networking.

In general, a cloud-based computing environment is a resource that typically combines the computational power of a large grouping of processors (such as within web servers) and/or that combines the storage capacity of a large grouping of computer memories or storage devices. Systems that provide cloud-based resources may be utilized exclusively by their owners or such systems may be accessible to outside users who deploy applications within the computing infrastructure to obtain the benefit of large computational or storage resources.

The cloud is formed, for example, by a network of web servers that comprise a plurality of computing devices, such as the host machine 4002, with each server 4030 (or at least a plurality thereof) providing processor and/or storage resources. These servers manage workloads provided by multiple users (e.g., cloud resource customers or other users). Typically, each user places workload demands upon the cloud that vary in real-time, sometimes dramatically. The nature and extent of these variations typically depends on the type of business associated with the user.

It is noteworthy that any hardware platform suitable for performing the processing described herein is suitable for use with the technology. The terms “computer-readable storage medium” and “computer-readable storage media” as used herein refer to any medium or media that participate in providing instructions to a CPU for execution. Such media can take many forms, including, but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical or magnetic disks, such as a fixed disk. Volatile media include dynamic memory, such as system RAM. Transmission media include coaxial cables, copper wire and fiber optics, among others, including the wires that comprise one aspect of a bus. Transmission media can also take the form of acoustic or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include, for example, a flexible disk, a hard disk, magnetic tape, any other magnetic medium, a CD-ROM disk, digital video disk (DVD), any other optical medium, any other physical medium with patterns of marks or holes, a RAM, a PROM, an EPROM, an EEPROM, a FLASH EPROM, any other memory chip or data exchange adapter, a carrier wave, or any other medium from which a computer can read.

Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to a CPU for execution. A bus carries the data to system RAM, from which a CPU retrieves and executes the instructions. The instructions received by system RAM can optionally be stored on a fixed disk either before or after execution by a CPU.

Computer program code for carrying out operations for aspects of the present technology may be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++, or the like and conventional procedural programming languages, such as the “C” programming language, Go, Python, or other programming languages, including assembly languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Examples of the method according to various aspects of the present disclosure are provided below in the following numbered clauses. An aspect of the method may include any one or more than one, and any combination of, the numbered clauses described below.

FIG. 6 illustrates one example of the inputs and outputs of automated page selection from a digital data record, according to at least one aspect of the present disclosure. With reference now primarily to FIG. 6 together with FIG. 1-2 , examples 600 include inputs 601 and outputs 602 which may be input into page selection 103, FIG. 1 or selection of page number 205, FIG. 2 . The outputs are produced by page selection 103, FIG. 1 or selection of page number 205, FIG. 2 . For example, inputs 603, and 605 may be any type of digital data record such as an SDS document, and may be in various document formats, for example PDF. The page number or output 602 provides the page numbers with relevant or target information of inputs 601. The output 604 of document 603 contains two page numbers as more than one page number is identified as containing target or relevant information. Document 605 however, only contains one page that is identified as containing relevant or target information and therefore output 606 contains a reference to only one page number.

FIG. 7A-7B illustrates one example of the inputs and outputs of image generation from a digital data record, according to at least one aspect of the present disclosure. With reference now primarily to FIG. 7A-7B together with FIGS. 1-2 and 6 , example 700 may input an extracted document 701 and page numbers 702, which may correspond to document 603, and identified output page numbers 604, FIG. 6 . In this example the identified page numbers 702 as pages [2, 3] of document 701, this information, including the page numbers may be equivalent to page number 104, FIG. 1 for example. These page numbers may therefore be input 210, FIG. 2 into an image generation process or module 106, FIG. 1 along with document 101, FIG. 1 which corresponds to document 701. Image generation process or module 106, FIG. 1 may then output images 107, FIG. 1 , which correspond to images 703 and 704. In this example two page numbers 702 are identified, and therefore images of two separate pages are produced. However in other examples where only one page is identified then only one image of that particular page is generated.

FIG. 8 illustrates one example of the inputs and outputs of a detection of sections, tables, or portions of the image(s), according to at least one aspect of the present disclosure. With reference now primarily to FIG. 8 together with FIGS. 1-2, and 7-8 , example 800 may include inputs of page images 703, 704, FIG. 7A-7B that were generated at example 700 or 106, FIG. 1 for example. These image inputs are fed into a CV model 108, FIG. 1 , so that the model 108 may detect the relevant sections in the images. The output of CV model 108, FIG. 1 may be separate/independent outputs for each page image 703, 704 that is input into CV model 108. For example, for page image 703 the output coordinates CV model 108 are coordinates 801 of a relevant section, in this example an identified table 802, containing relevant chemical information.

FIG. 9 illustrates one example of the inputs and outputs of extraction of sections of an image, according to at least one aspect of the present disclosure. With reference now primarily to FIG. 9 together with FIGS. 1-2, and 7-8 , example 900 may include various inputs 901 that are input 220, FIG. 2 or provided to a table or section extraction module 110 such as a pdf extraction module, which may extract an area, for example area 802, or 804, FIG. 8 from a generated image, for example image 703, or 704, FIG. 7A-7B. The various inputs may include the document 901, 101, page numbers 902, 903, 104 and images 703, 704, FIGS. 7A-7B and 107 , FIG. 1 , as well as coordinates 903, 904, 109, FIG. 1 that may be derived from CV model 108, FIG. 1 . The output(s) 905, 906 may comprise of a data frame 111, FIG. 1 such as a table that is extracted 225, FIG. 2 by the extraction module. The table may contain text, numbers or other information such as names, compositions, weights, and percentages.

FIG. 10 illustrates one example of the inputs and outputs of extraction of data, information, or text from extracted portions of an image, according to at least one aspect of the present disclosure. With reference now primarily to FIG. 10 together with FIGS. 1-2, and 9 , example 1000 may include an input 1001, 1002 which may contain extracted tables, or sections from one or more images, these inputs 1001, 1002 may correspond to data frame 111, FIG. 1 , that were produced as outputs 905, 906, FIG. 9 . Once these inputs 1001, 1002 are provided to a target information extraction module 112, FIG. 1 , then information or data 1003 may be output, which may include names, compositions, weights, numbers, or percentages as examples.

FIG. 11 illustrates a flow chart of one aspect of a method to automatically extract indexing fields from a digital data record, according to at least one aspect of the present disclosure. Method 1100 may include receiving 1105 a digital data record such as an SDS. The document may be received in any format, including an image format such as a PDF or PNG document. Method 1100 continues to convert 1110 the data or contents in the digital data record into a text format. This is not a simple conversion, but rather is a strategic conversion, aimed to maximize the efficiency of the ultimate pipeline. For example a document can first be checked to determine whether it is an SDS by using rule based approaches using section headers and label headers it contains. Then if the document is determined to be an SDS, a predetermined number of pages and/or pages with specific locations in the document (for example the first two pages of the document are checked for specific field(s)), in other embodiments, the first two and last two pages, can be checked for some fields, or for composition section(s) of content field(s). This maximizes the efficiency because only some pages and fields are then converted. Different variations of these texts may be created for each field for maximum accuracy. Once the document is converted to a text format, preprocessing 1115 is undertaken on the converted text to make it easier for machine learning models to extract specific information that's required. Various rules can be implemented to deal with the inconsistencies in SDS formats and templates, and the use-case specific rules which involve differences in the manner text appear on SDS vs text to be indexed.

Various fields and types of contents can be extracted, identified and indexed by method 1100, these may include for example names, identifiers, types, dates, codes, aliases and the like. In preferred embodiments, where SDS documents are geared towards chemical enterprises a list of fields to be indexed could include product name(s), product code(s), manufacturer name(s), supplier, revision date, CAS or other identifier codes for chemicals, aliases, languages, and regulatory formats or regions. In various embodiments, for at least one field, the pre-trained machine learning model is used to identify the content belonging to the specific field. Therefore if there are six fields for example, in some embodiments six separate machine learning models are used where the text is fed into six separate models 1120, where each machine learning model 1120 is designed to identify and extract one type of content belonging to a specific field. In one example these preprocessed versions of texts are then fed into a number of natural language processing (NLP) machine learning models, for example six models for six fields, where potential index candidates are created. Example fields where a machine learning model is used to identify and extract content can include for example Product Name, Product Code, CAS, Alias, Manufacturer Name, and Supplier Name are the six fields that use machine learning models. These machine learning models are created and trained in a separate process by using vast quantities of Safety Data Sheets that were historically indexed using human labor. In several aspects, for different field types, a separate process can be used, for example the process described in FIG. 1 .

Each model 1120 that is deployed as part of method 1100 outputs identified and extracted content or text from its associated or specific field. These output(s) or candidate output(s) are then post-processed 1125. At this stage each potential candidate again goes through additional scrutiny and checks to ensure the validity of these candidate outputs. Final inferences are then obtained for each field, out of these potential candidates. In addition, further post-processing can be done to match the raw model inferences with exact values to be seen, this could include simple presentation level changes (such as matching a case of a letter or punctuation back to what is on the document), or it could include complex changes may be matched such as a manufacturer name from SDS text to what is in the database (which may not be identical). Some of these changes involve simple changes like matching case, punctuation, etc., and some use-case specific changes like Manufacturer and Supplier nuances. Finally, indexes 1130 can be produced which can then be displayed on a display screen or other display device.

FIG. 12 illustrates a flow diagram of one aspect of a method to automatically extract indexing fields from a digital data record as well as extracting specialized information via a separate pipeline, according to at least one aspect of the present disclosure. Method 1200 can be implemented on a digital data record 1205, of various formats, for example an image format such as a PDF. A page selection module 1210 may be run on digital data record 1205, for example similar to page selection module 103, FIG. 1 . The page selection module 1210 may be run or initiated multiple times on a digital data record 1205, with each time it is run to identify pages related to a specific field or content type in the digital data record. Alternatively the page selection module may be run to identify various or multiple field or content types, and output the page number(s) for each of these types. Page selection module 1210 takes the digital data record as input, for example in PDF format, and uses rule-based approaches to identify which page(s) within the SDS pages contains the information that is being sought, and returns or outputs page numbers. This step restricts the search to the pages containing the required information and improves the system's overall precision as discussed above to maximize the efficiency of the machine learning models utilized.

In some aspects, running page selection processes via the page selection module 1210 is not necessary, for example when page numbers where text of target content or field types in the digital data record 1205 are known. In these instances the digital data record and known page numbers are directly input 1215 into the text extraction module 1220 to begin text extraction from the specified pages of digital data record 1205.

Once pages are selected, for example by page selection module 1210, then text is extracted from these pages, for example by a text extraction module 1220. In several aspects, text is extracted from each page that is identified. The text extraction module 1220 receives the output page numbers from the page selection module 1210, along with the digital data record 1205 in its original format as input, and extracts the text from the specified pages of the digital data record 1205. The text extracted is passed on, for example as a long string, to a next module. Various and multiple versions or copies of text may be extracted, due to differences in page numbers returned, for each of the fields where information is being extracted from.

In several aspects, once text is extracted by text extraction module 1220, then the text is pre-processed by an automated pre-processing module 1225. These pre-processing algorithms are designed to produce text outputs that are more easily processed by the machine learning model to extract the specific information that's required. The pre-processing steps include, but are not limited to substitutions: for example some of the redundant information is substituted with blank string in the extracted text; specific information removal, for example contact info or address removal where any type of contact information such as email address, phone number, and website are removed from the text as they do not provide any useful context for detecting the standard indexing fields, removal of any kind of address, company headquarter information, and the like can also be removed; change extensions, for example some extensions or abbreviations in the company name are removed or abbreviated; finally other general text preprocessing that is required for any text-based tasks can also be undertaken.

Similar to the page selection module 1210 and the text extraction module 1220, the text preprocessing module 1225 may also return or output multiple copies of processed or cleaned text, due to differences in processing rules and page numbers extracted, for each of the fields or content types being extracted. For each pre-determined field, or content type in the digital data record 1205 that is preprocessed, the preprocessed text of that field is input 1230 into a pre-trained machine learning model that is designed to accept inputs of preprocessed text and output 1235 the content that belongs to that specific field type. For SDS documents, the field types can include and are not limited to CAS code(s), Manufacturer Name, Supplier Name, Product Name, Product Code, Alias, Revision Date, Language, and Regulatory Format. For each of these fields, the machine learning network has been pre-trained on mass quantities of documents in a supervised or unsupervised setting depending on the field or content type and the available training data.

The output 1235 produced by each pre-trained machine learning model can include individual inferences for each field. Multiple outputs, or inferences may be produced or output 1235 by each network, whereby some are relevant and others may not be. Verification and validation checks can also be applied to either select from candidate outputs or verify and validate the accuracy or relevance of these outputs. After all validation checks are passed, the indexes (or model inferences) from each field are combined and returned and saved in the database 1240. In several aspects, these indexes may be displayed on a graphical user interface or in a list format for an end-user.

In several embodiments, one or more field types are determined separately to or with processes that are in parallel to the processes 1220-1235, via one or more other modules targeted towards differently formatted or different types of content in digital data record 1205. For example a certain code or number like a CAS code or number formatted in a manner that is too difficult to extract using the steps 1220-1235. In one example a CAS code, or chemical compositions or ingredients are extracted and identified by one or more modules 1245. Modules 1245 can comprise one or more processes 100-300 described in any of FIG. 1-3 for example.

FIG. 13 illustrates a method of extracting and indexing content from a digital data record, according to at least one aspect of the present disclosure. In one aspect, the method 1300 includes identifying 1305, based on rules defining target information fields, for each target field of the target information fields, at least one page in a digital data record comprising content related to the target field. These target information fields may be pre-set or pre-determined for example based on rules for each type of digital data record 1205, FIG. 12 . This could be carried out for example by page selection module 1210, FIG. 12 . Once a page or page number if identified 1305, then method 1300 includes extracting, for each target field, from the identified at least one page, at least one portion of text comprising the content. This means that for example in a field of ‘product name’.

The method 1300 continues with extracting, 1310, for each target field, from the identified at least one page, at least one portion of text comprising the content, which can be for example undertaken by a text extraction module 1220, FIG. 12 . For each portion of text that is extracted for each target field, method 1300 can continue to feed 1315 for each target field, a pre-processed version of the at least one portion of text into a machine learning (ML) model, wherein the ML model is trained on the target field. In several aspects, these portions of text that are extracted can be pre-processed and then fed 1315 into the ML model.

In several aspects, the ML models that received the preprocessed text can then determine 1320, for each target field, via the ML model trained on the target field, at least one candidate text comprising the content. Finally, for each target field, extract 1325 the at least one candidate text. In several aspects, multiple candidate texts may be extracted 1325, and then one or more multiple candidate texts can be chosen after validation or post-processing.

FIG. 14 illustrates one example of inputs and outputs of page selection of a digital data record, according to at least one aspect of the present disclosure. Document 1400 can include an input digital data record, for example an SDS document, with document name 1405. Document 1400 can be input into page selection module 1210, FIG. 12 which depending on the content or field type out puts one or more page numbers. In this example the page selection module 1210 outputs page numbers 1410 for field types ‘Product Name’, ‘Manufacturer Name’, ‘Supplier Name’, and ‘Alias’. Page selection module 1210 also outputs page numbers 1415 for ‘Product Code’ and ‘Revision Date’.

FIG. 15 illustrates one example of inputs and outputs of text extraction of a digital data record, according to at least one aspect of the present disclosure. Page numbers 1410, and 1415, FIG. 14 can then be input into a text extraction module, for example text extraction module 1220, FIG. 12 . For example page number 1510 are input along with digital data record 1505 into a text extraction module, which then produces outputs of text portions for various fields. Text 1515 may be extracted for several fields including product name, manufacturer name, supplier name, and alias. Text 1520 may also be extracted for other predetermined field types or content.

FIG. 16 illustrates one example of inputs and outputs of preprocessing of text in a digital data record, according to at least one aspect of the present disclosure. Example 1600 includes text that was extracted, such as text 1515, or 1520, FIG. 15 , as an input 1605 for preprocessing. This text is then preprocessed and output as preprocessed text 1610. The preprocessing may be undertaken for any and all portions of text that were extracted for all various field types and content types.

FIG. 17 illustrates one example of text inference output from preprocessed text in a digital data record, according to at least one aspect of the present disclosure. Example 1700 includes preprocessed text, for example preprocessed text 1610, as input 1705 into a pre-trained ML model. The pre-trained ML model then outputs an inference text 1710. In several aspects, a number of candidate inference texts are output. Example 1700 comprises an ML model trained for product names. It takes an input text 1705, and outputs the product name 1710. Different models may receive the same input text 1705, and output different results, for example manufacturer names, dates, places, and any other type of content of field/category the ML model is trained on.

FIG. 18 illustrates another example of text inference output from preprocessed text in a digital data record, according to at least one aspect of the present disclosure. Example 1800 is another example of text input 1805 being input into a specifically trained ML model to produce output 1810 of inference text, in this instance an inference text of a revision data and language. Input text 1805, uses the same preprocessed text as input 1705, FIG. 17 but produces different output 1810 to output 1710 based on the ML model used in example 1800 trained to produce inferences of language classification and date classification as output 1810.

FIG. 19 illustrates one example of the inferences output by the methods disclosed herein to be combined and displayed, according to at least one aspect of the present disclosure. Inputs 1905 are output as indexes 1910, and may be displayed to a user in a graphical user interface.

In several embodiments, this block of input text 1905 is run through a verification block to provide a form of quality assurance and to produce output indexes 1910. This can be done proceeding step application of models 1120, FIG. 11 , or as part of post-processing 1125, FIG. 11 of the text, in generating indexes 1130 Input text may be classified into two categories, ‘affirmation and ‘elimination’ text, These two types of texts may be used as prompt when they occur nearby an indexed field. For example, if elimination text is present the output would not return the results as the presence of elimination text indicates that the field is misleading and mistakenly chosen by the ML model. The presence of affirming text means to return the value found by the ML model in 1120, FIG. 11 . These categories of text and specifically classifying them into prompting texts of ‘affirmation’ or ‘elimination’ may be limited to specific types of fields and/or their corresponding ML models.

The verification block may also include a formatting check to determine if the length of the output of the model is of reasonable length, with certain thresholds for each field that can be set at different minimum and maximal lengths within the range of what is acceptable. For example if the indexed values contain only numbers or special characters—which would be flagged as not containing enough alphabetical characters. This would be the case as well if the number of alphabetical characters was lower than a pre-set or predetermined threshold that could be manually or automatically set. Examples of the methods and systems according to various aspects of the present disclosure are provided below in the following numbered clauses. An aspect of the method or system may include any one or more than one, and any combination of, the numbered clauses described below.

-   -   Clause 1. A method to automatically index multiple informational         fields in a digital data record, the method comprising:         identifying, based on rules defining target information fields,         for each target field of the target information fields, at least         one page in a digital data record comprising content related to         the target field; extracting, for each target field, from the         identified at least one page, at least one portion of text         comprising the content; feeding, for each target field, a         pre-processed version of the at least one portion of text into a         machine learning (ML) model, wherein the ML model is trained on         the target field; determining, for each target field, via the ML         model trained on the target field, at least one candidate text         comprising the content; and extracting, for each target field,         the at least one candidate text.     -   Clause 2. The method of Clause 1, wherein the candidate text is         empty and does not contain alphanumerical characters, symbols or         other characters.     -   Clause 3. The method of any of Clauses 1-2, further comprising         identifying, based on rules defining the content, the at least         one portion of text comprising the content within the at least         one page.     -   Clause 4. The method of any of Clauses 1-3, further comprising:         validating the at least one candidate text.     -   Clause 5. The method of of any of Clauses 1-4, further         comprising generating, for each target field, an index based on         the at least one candidate text.     -   Clause 6. The method of of any of Clauses 1-5, further         comprising displaying the generated at least one candidate text         for each target field on a graphical user interface.     -   Clause 7. The method of of any of Clauses 1-6, further         comprising combining, the extracted at least one candidate text         from each target field.     -   Clause 8. The method of of any of Clauses 1-7, further         comprising saving, for each target field, the at least one         candidate text into a database.     -   Clause 9. The method of of any of Clauses 1-8, wherein the         extracting of the at least one portion of text comprising the         content comprises converting for each target field, from the         identified at least one page, the at least one portion of text         comprising the content from an image format into a text format.     -   Clause 10. The method of of any of Clauses 1-9, wherein the text         format is a string.     -   Clause 11. The method of of any of Clauses 1-10, further         comprising: identifying a page of the digital data record         containing special target information; inputting an image of the         page into a visual machine learning network (visual ML), wherein         the visual ML is trained to recognize text associated with the         special target information in an image; identifying by the         visual ML, a section of the image that contains the special         target information; inputting a page number of the page, the         digital data record, and coordinates of the section into an         extraction module; and extracting the special target information         by the extraction module from the section.     -   Clause 12. The method of of any of Clauses 1-11 further         comprising inputting the extracted special target information         into a natural language processing ML network (NLP); and based         on a presence of at least one data item in the extracted special         target information, identifying at least one data item, by the         NLP model, based on a structure of the extracted special target         information.     -   Clause 13. A system to automatically index multiple         informational fields in a digital data record, the system         comprising: a plurality of trained NLP ML models, wherein each         ML model of the plurality of trained NLP models is trained on a         specific target information field of a plurality of target         information fields; a database to store indexed information; a         memory comprising instructions, coupled to a processor, wherein         the instructions are executable by a processor to: identify,         based on rules defining the target information fields, for each         target field of the target information fields, at least one page         in the digital data record comprising content related to the         target field; extract, for each target field, from the         identified at least one page, at least one portion of text         comprising the content; feed, for each target field, a         pre-processed version of the at least one portion of text into         an ML model of the plurality of trained NLP ML models, wherein         the ML model is trained on the target field; determine, for each         target field, via the ML model trained on the target field, at         least one candidate text comprising the content; extract, for         each target field, via the ML model trained on the target field,         the at least one candidate text; and generate for each target         field, via the ML model trained on the target field, an index of         the at least one candidate text.     -   Clause 13. The system of of Claus 12, wherein the instructions         further comprise validate, for each target field, the at least         one candidate text.     -   Clause 14. The system of any of Clauses 12-13, wherein the         instructions further comprise display the index of the at least         one candidate text on a graphical user interface.     -   Clause 15. The system of any of Clauses 12-14, further         comprising combine the index of the at least one candidate text,         with at least one other index of an at least one candidate index         of at least one other target field of the plurality of target         information fields.     -   Clause 16. The system any of Clauses 12-15, further comprising         saving, for each target field, the generated index of the at         least one candidate text into the database.     -   Clause 17. The system of any of Clauses 12-16, further         comprising: identifying a page of the digital data record         containing specialized information; inputting an image of the         page into a visual machine learning network (visual ML), wherein         the visual ML is trained to recognize text associated with the         specialized information in an image; identifying by the visual         ML, a section of the image that contains the specialized         information; inputting a page number of the page, the digital         data record, and coordinates of the section into an extraction         module; and extracting the specialized information by the         extraction module from the section.     -   Clause 18. The system of any of Clauses 12-17, further         comprising inputting the extracted specialized information into         a natural language processing ML network (NLP); and based on a         presence of at least one data item in the extracted specialized         information, identifying at least one data item, by the NLP         model, based on a structure of the extracted specialized         information.     -   Clause 19. A non-transitory computer-readable storage medium         having embodied thereon a program, the program being executable         by a processor to perform a method for providing a sentiment for         digital information comprising: receiving a digital data record;         identifying, based on rules defining target information fields,         for each target field of the target information fields, at least         one page in the digital data record comprising content related         to the target field; extracting, for each target field, from the         identified at least one page, at least one portion of text         comprising the content; feeding, for each target field, a         pre-processed version of the at least one portion of text into a         machine learning (ML) model, wherein the ML model is trained on         the target field; determining, for each target field, via the ML         model trained on the target field, at least one candidate text         comprising the content; and extracting, for each target field,         the at least one candidate text.     -   20. The non-transitory computer-readable storage medium of         Clause 19, wherein the program being executable by the processor         further performs: identifying a page of the digital data record         containing specialized information; inputting an image of the         page into a visual machine learning network (visual ML), wherein         the visual ML is trained to recognize text associated with the         specialized information in an image; identifying by the visual         ML, a section of the image that contains the specialized         information; inputting a page number of the page, the digital         data record, and coordinates of the section into an extraction         module; extracting the specialized information by the extraction         module from the section; inputting the extracted specialized         information into a natural language processing ML network (NLP);         and based on a presence of at least one data item in the         extracted specialized information, identifying at least one data         item, by the NLP model, based on a structure of the extracted         specialized information.

The foregoing detailed description has set forth various forms of the systems and/or processes via the use of block diagrams, flowcharts, and/or examples. Insofar as such block diagrams, flowcharts, and/or examples contain one or more functions and/or operations, it will be understood by those within the art that each function and/or operation within such block diagrams, flowcharts, and/or examples can be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. Those skilled in the art will recognize that some aspects of the forms disclosed herein, in whole or in part, can be equivalently implemented in integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware would be well within the skill of one of skill in the art in light of this disclosure. In addition, those skilled in the art will appreciate that the mechanisms of the subject matter described herein are capable of being distributed as one or more program products in a variety of forms, and that an illustrative form of the subject matter described herein applies regardless of the particular type of signal bearing medium used to actually carry out the distribution.

Instructions used to program logic to perform various disclosed aspects can be stored within a memory in the system, such as dynamic random access memory (DRAM), cache, flash memory, or other storage. Furthermore, the instructions can be distributed via a network or by way of other computer readable media. Thus a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), but is not limited to, floppy diskettes, optical disks, compact disc, read-only memory (CD-ROMs), and magneto-optical disks, read-only memory (ROMs), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, or a tangible, machine-readable storage used in the transmission of information over the Internet via electrical, optical, acoustical or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.). Accordingly, the non-transitory computer-readable medium includes any type of tangible machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

Any of the software components or functions described in this application, may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Python, Java, C++ or Perl using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions, or commands on a computer readable medium, such as RAM, ROM, a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a CD-ROM. Any such computer readable medium may reside on or within a single computational apparatus, and may be present on or within different computational apparatuses within a system or network.

As used in any aspect herein, the term “logic” may refer to an app, software, firmware and/or circuitry configured to perform any of the aforementioned operations. Software may be embodied as a software package, code, instructions, instruction sets and/or data recorded on non-transitory computer readable storage medium. Firmware may be embodied as code, instructions or instruction sets and/or data that are hard-coded (e.g., nonvolatile) in memory devices.

As used in any aspect herein, the terms “component,” “system,” “module” and the like can refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution.

As used in any aspect herein, an “algorithm” refers to a self-consistent sequence of steps leading to a desired result, where a “step” refers to a manipulation of physical quantities and/or logic states which may, though need not necessarily, take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It is common usage to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. These and similar terms may be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities and/or states.

A network may include a packet switched network. The communication devices may be capable of communicating with each other using a selected packet switched network communications protocol. One example communications protocol may include an Ethernet communications protocol which may be capable of permitting communication using a Transmission Control Protocol/Internet Protocol (TCP/IP). The Ethernet protocol may comply or be compatible with the Ethernet standard published by the Institute of Electrical and Electronics Engineers (IEEE) titled “IEEE 802.3 Standard”, published in December, 2008 and/or later versions of this standard. Alternatively or additionally, the communication devices may be capable of communicating with each other using an X.25 communications protocol. The X.25 communications protocol may comply or be compatible with a standard promulgated by the International Telecommunication Union-Telecommunication Standardization Sector (ITU-T). Alternatively or additionally, the communication devices may be capable of communicating with each other using a frame relay communications protocol. The frame relay communications protocol may comply or be compatible with a standard promulgated by Consultative Committee for International Telegraph and Telephone (CCITT) and/or the American National Standards Institute (ANSI). Alternatively or additionally, the transceivers may be capable of communicating with each other using an Asynchronous Transfer Mode (ATM) communications protocol. The ATM communications protocol may comply or be compatible with an ATM standard published by the ATM Forum titled “ATM-MPLS Network Interworking 2.0” published August 2001, and/or later versions of this standard. Of course, different and/or after-developed connection-oriented network communication protocols are equally contemplated herein.

Unless specifically stated otherwise as apparent from the foregoing disclosure, it is appreciated that, throughout the present disclosure, discussions using terms such as “processing,” “computing,” “calculating,” “determining,” “displaying,” or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

One or more components may be referred to herein as “configured to,” “configurable to,” “operable/operative to,” “adapted/adaptable,” “able to,” “conformable/conformed to,” etc. Those skilled in the art will recognize that “configured to” can generally encompass active-state components and/or inactive-state components and/or standby-state components, unless context requires otherwise.

Those skilled in the art will recognize that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes but is not limited to,” etc.). It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases “at least one” and “one or more” to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to claims containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and/or “an” should typically be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations.

In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of “two recitations,” without other modifiers, typically means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to “at least one of A, B, and C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, and C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). In those instances where a convention analogous to “at least one of A, B, or C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, or C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). It will be further understood by those within the art that typically a disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms unless context dictates otherwise. For example, the phrase “A or B” will be typically understood to include the possibilities of “A” or “B” or “A and B.”

With respect to the appended claims, those skilled in the art will appreciate that recited operations therein may generally be performed in any order. Also, although various operational flow diagrams are presented in a sequence(s), it should be understood that the various operations may be performed in other orders than those which are illustrated, or may be performed concurrently. Examples of such alternate orderings may include overlapping, interleaved, interrupted, reordered, incremental, preparatory, supplemental, simultaneous, reverse, or other variant orderings, unless context dictates otherwise. Furthermore, terms like “responsive to,” “related to,” or other past-tense adjectives are generally not intended to exclude such variants, unless context dictates otherwise.

It is worthy to note that any reference to “one aspect,” “an aspect,” “an exemplification,” “one exemplification,” and the like means that a particular feature, structure, or characteristic described in connection with the aspect is included in at least one aspect. Thus, appearances of the phrases “in one aspect,” “in an aspect,” “in an exemplification,” and “in one exemplification” in various places throughout the specification are not necessarily all referring to the same aspect. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more aspects.

As used herein, the singular form of “a”, “an”, and “the” include the plural references unless the context clearly dictates otherwise.

As used herein, the term “comprising” is not intended to be limiting, but may be a transitional term synonymous with “including,” “containing,” or “characterized by.” The term “comprising” may thereby be inclusive or open-ended and does not exclude additional, unrecited elements or method steps when used in a claim. For instance, in describing a method, “comprising” indicates that the claim is open-ended and allows for additional steps. In describing a device, “comprising” may mean that a named element(s) may be essential for an embodiment or aspect, but other elements may be added and still form a construct within the scope of a claim. In contrast, the transitional phrase “consisting of” excludes any element, step, or ingredient not specified in a claim. This is consistent with the use of the term throughout the specification.

Any patent application, patent, non-patent publication, or other disclosure material referred to in this specification and/or listed in any Application Data Sheet is incorporated by reference herein, to the extent that the incorporated materials is not inconsistent herewith. As such, and to the extent necessary, the disclosure as explicitly set forth herein supersedes any conflicting material incorporated herein by reference. Any material, or portion thereof, that is said to be incorporated by reference herein, but which conflicts with existing definitions, statements, or other disclosure material set forth herein will only be incorporated to the extent that no conflict arises between that incorporated material and the existing disclosure material. None is admitted to be prior art.

In summary, numerous benefits have been described which result from employing the concepts described herein. The foregoing description of the one or more forms has been presented for purposes of illustration and description. It is not intended to be exhaustive or limiting to the precise form disclosed. Modifications or variations are possible in light of the above teachings. The one or more forms were chosen and described in order to illustrate principles and practical application to thereby enable one of ordinary skill in the art to utilize the various forms and with various modifications as are suited to the particular use contemplated. It is intended that the claims submitted herewith define the overall scope. 

What is claimed is:
 1. A method to automatically index multiple informational fields in digital data records, the method comprising: identifying, based on rules defining target information fields, for each target field of the target information fields, at least one page in a digital data record comprising content related to the target field, wherein the identifying comprises outputting at least one page number of the at least one page; extracting, for each target field, from the at least one page, at least one portion of text comprising the content; feeding, for each target field, a pre-processed version of the at least one portion of text into a machine learning (ML) model, wherein the ML model is trained on the target field; determining, for each target field, via the ML model trained on the target field, at least one candidate text comprising the content; and extracting, for each target field, the at least one candidate text.
 2. The method of claim 1, wherein the candidate text is empty and does not contain alphanumerical characters, symbols or other characters.
 3. The method of claim 1, further comprising: identifying, based on rules defining the content, the at least one portion of text comprising the content within the at least one page.
 4. The method of claim 1, further comprising: validating the at least one candidate text.
 5. The method of claim 1, further comprising: generating, for each target field, an index based on the at least one candidate text.
 6. The method of claim 4, further comprising: displaying the at least one candidate text for each target field on a graphical user interface.
 7. The method of claim 1, further comprising: combining, the at least one candidate text from each target field.
 8. The method of claim 1, further comprising: saving, for each target field, the at least one candidate text into a database.
 9. The method of claim 1, wherein the extracting of the at least one portion of text comprising the content comprises: converting for each target field, from the at least one page, the at least one portion of text comprising the content from an image format into a text format.
 10. The method of claim 9, wherein the text format is a string.
 11. A computer implemented method comprising: identifying, based on rules defining target information fields, for each target field of the target information fields, at least one page in a digital data record comprising content related to the target field; extracting, for each target field, from the at least one page, at least one portion of text comprising the content; feeding, for each target field, a pre-processed version of the at least one portion of text into a machine learning (ML) model, wherein the ML model is trained on data associated with the target field; determining, for each target field, via the ML model trained on the target field, at least one candidate text comprising the content; extracting, for each target field, the at least one candidate text; identifying a page of the digital data record containing special target information; inputting an image of the page into a visual machine learning network (visual ML), wherein the visual ML is trained to recognize text associated with the special target information in an image; identifying by the visual ML, a section of the image that contains the special target information; inputting a page number of the page, the digital data record, and coordinates of the section into an extraction module; and extracting the special target information by the extraction module from the section.
 12. The method of claim 11, further comprising: inputting the special target information into a natural language processing ML network (NLP); and based on a presence of at least one data item in the special target information, identifying at least one data item, by the NLP, based on a structure of the special target information.
 13. A system to automatically index multiple informational fields in a digital data record, the system comprising: a plurality of trained NLP ML models, wherein each ML model of the plurality of trained NLP ML models is trained on a specific target information field of a plurality of target information fields; a database to store indexed information; a memory comprising instructions, coupled to a processor, wherein the instructions are executable by a processor to: identify, based on rules defining the target information fields, for each target field of the target information fields, at least one page in the digital data record comprising content related to the target field; extract, for each target field, from the at least one page, at least one portion of text comprising the content; feed, for each target field, a pre-processed version of the at least one portion of text into an ML model of the plurality of trained NLP ML models, wherein the ML model is trained on the target field; determine, for each target field, via the ML model trained on the target field, at least one candidate text comprising the content; extract, for each target field, via the ML model trained on the target field, the at least one candidate text; and generate for each target field, via the ML model trained on the target field, an index of the at least one candidate text.
 14. The system of claim 3, wherein the instructions further comprise: validate, for each target field, the at least one candidate text.
 15. The system of claim 3, wherein the instructions further comprise: display the index of the at least one candidate text on a graphical user interface.
 16. The system of claim 3, further comprising: combine the index of the at least one candidate text, with at least one other index of an at least one candidate index of at least one other target field of the plurality of target information fields.
 17. The system of claim 3, further comprising: saving, for each target field, the index of the at least one candidate text into the database.
 18. The system of claim 3, further comprising: identifying a page of the digital data record that contains specialized information; inputting an image of the page into a visual machine learning network (visual ML), wherein the visual ML is trained to recognize text associated with the specialized information; identifying by the visual ML, a section of the image that contains the specialized information; inputting a page number of the page, the digital data record, and coordinates of the section into an extraction module; and extracting the specialized information by the extraction module from the section.
 19. The system of claim 18, further comprising: inputting the specialized information into a natural language processing ML network (NLP); and based on a presence of at least one data item in the specialized information, identifying at least one data item, by the NLP, based on a structure of the specialized information.
 20. A non-transitory computer-readable storage medium having embodied thereon a program, the program executable by a processor to perform a method for automatically indexing multiple informational fields in a digital data record comprising: inputting at least one page identifier of at least one page of a digital data record, into an extraction module, wherein the at least one page identifier comprises at least one of a page number or other page identifier; extracting, by the extraction module, for each target field, from the at least one page, wherein the at least one page is based on at least the one page identifier, at least one portion of text comprising content related to a specific target information field; feeding, for each target field, a pre-processed version of the at least one portion of text into a machine learning (ML) model, wherein the ML model is trained on the target field; determining, for each target field, via the ML model trained on the target field, at least one candidate text comprising the content; and extracting, for each target field, the at least one candidate text.
 21. The non-transitory computer-readable storage medium of claim 20, wherein the program further performs: identifying a page of the digital data record that contains specialized information; inputting an image of the page into a visual machine learning network (visual ML), wherein the visual ML is trained to recognize text associated with the specialized information; identifying by the visual ML, a section of the image that contains the specialized information; inputting a page number of the page, the digital data record, and coordinates of the section into an extraction module; extracting the specialized information by the extraction module from the section; inputting the specialized information into a natural language processing ML network (NLP); and based on a presence of at least one data item in the specialized information, identifying at least one data item, by the NLP, based on a structure of the specialized information. 